{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lab 26 - k-Nearest Neighbors classifier 2\n",
    "\n",
    "We will continue using the Titanic training and test data from [Kaggle](https://www.kaggle.com/c/titanic) from Lab 24 and 25.\n",
    "\n",
    "First import the necessary libraries."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import pandas as pd\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "from sklearn.model_selection import train_test_split\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Loading and cleaning the training data\n",
    "\n",
    "The code for loading in the data and cleaning it from Lab 25 is below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "train = pd.read_csv(\"../Data/train.csv\")\n",
    "train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# fill in missing age data\n",
    "train[\"Age\"] = train[\"Age\"].fillna(train[\"Age\"].median())\n",
    "\n",
    "# fill in the missing embarked data\n",
    "train[\"Embarked\"] = train[\"Embarked\"].fillna(\"S\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# create dummy variables for passenger class, sex, and embarked\n",
    "train2 = pd.get_dummies(train, columns = [\"Pclass\",\"Sex\",\"Embarked\"], drop_first = True)\n",
    "train2.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# remove the remaining qualitative columns\n",
    "train2.drop(\"Cabin\",axis = 1,inplace = True)\n",
    "train2.drop(\"Name\",axis = 1,inplace = True)\n",
    "train2.drop(\"Ticket\",axis = 1,inplace = True)\n",
    "\n",
    "# we should also drop PassengerId, although we did not last lab\n",
    "train2.drop(\"PassengerId\",axis = 1, inplace = True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# split the original training data into training and sets sets\n",
    "X_train,X_test,y_train, y_test =train_test_split(train2.drop(\"Survived\",axis=1),train2[\"Survived\"],test_size = 0.2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create a 3-nearest neighbor classifirer\n",
    "knn = KNeighborsClassifier(n_neighbors=3)\n",
    "# fit the classifier to the training data\n",
    "knn.fit(X_train, y_train)\n",
    "# test and score the classifier on our test data (part of the original training data)\n",
    "knn.score(X_test, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we are going to try running our classifier on the Kaggle test data and use all of our training data to fit the classifier.\n",
    "\n",
    "### Loading and cleaning the Kaggle test data\n",
    "\n",
    "First, load the test data from Kaggle into the dataframe `test`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have to process the test data in the same way as the training data, namely filling in the missing age and embarked data, creating the dummy variables for Pclass, Sex, and Embarked, and dropping the Cabin, Name, and Ticket columns.  Do this below, adding as many extra cells as you need.\n",
    "\n",
    "Call the processed dataframe `test2`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Store the column `PassengerId` in a variable.  We need this information for submitting our predictions to Kaggle, but don't want to use it in making the pedictions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Remove (drop) the `PassengerId` column from the test data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details> <summary>Pattern:</summary>\n",
    "<code>df = df.drop(\"name_of_column_to_drop\",axis=1)</code>\n",
    "</details>\n",
    "\n",
    "### Training the 3-nearest neighbor classifier\n",
    "\n",
    "Next we split up our training data into the answer (the `Survived` column) and the input data (all other columns).\n",
    "\n",
    "First, store the `Survived` column in the variable `y_train_kaggle`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, remove (drop) the `Survived` column from the training data, and store the new data frame in the variable `X_train_kaggle`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details> <summary>Pattern:</summary>\n",
    "<code>X_train_kaggle = df.drop(\"name_of_column_to_drop\",axis=1)</code>\n",
    "</details>\n",
    "\n",
    "Create a new k-nearest neighbors object with k = 3, and fit it on the entire training data (`X_train_kaggle`)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "knn_kaggle = KNeighborsClassifier(n_neighbors=3)\n",
    "knn_kaggle.fit(X_train_kaggle,y_train_kaggle)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details> <summary>Pattern:</summary>\n",
    "<code>knn_classifier_var_name = KNeighborsClassifier(n_neighbors= k)\n",
    "knn_classifier_var_name.fit(training_data,correct_output)\n",
    "</code>\n",
    "</details>\n",
    "\n",
    "### Making predictions with our 3-nearest neighbor classifier\n",
    "\n",
    "Now use this fitted classifier to make predictions on the test data (`test2`), and store it in the variable `y_pred`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred = knn_kaggle.predict(test2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The error message says something about NaN.  Could there be missing data (NaN) is a different column in the test data?  Use the `describe()` function to see if this is the case."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The fare column is missing one value.  Fill it in with the median fare."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details> <summary>Pattern:</summary>\n",
    "Assuming the variable `median_fare` contains the median fare:\n",
    "<code>df[\"Fare\"] = df[\"Fare\"].fillna(median_fare)\n",
    "</code>\n",
    "</details>\n",
    "\n",
    "Now try making the prediction again."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, we want to check our predictions by uploading them to Kaggle.  Kaggle wants the predictions in a CSV file with two columns:  PassengerID and Survived (our predictions).\n",
    "\n",
    "First we will create a new dataframe containing these two columns.  The code below assumes you stored the passenger ID column in the variable `passengerID`, but you can change this to whatever variable name you used."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "df = pd.DataFrame(data = {\"PassengerId\":passengerId, \"Survived\":y_pred})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next create a new CSV containing the data in this dataframe:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "df.to_csv(\"test1.csv\",index = 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Can you see the CSV file in your directory?  Download it and try uploading it to Kaggle (you will have to create an account) to check your predictions.  [Submission site](https://www.kaggle.com/c/titanic/submit)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}